Method and apparatus for two-dimensional finger motion tracking and control

ABSTRACT

Enhanced accuracy finger position and motion sensors devices, algorithms, and methods are disclosed that can be used in a variety of different applications. The sensors can be used in conjunction with partial fingerprint imagers to produce improved fingerprint scanners. The finger motion sensors may also be used (either with or without a partial fingerprint imager) to control electronic devices. When several of these finger motion and position sensors are aligned in different directions, finger motion over a two dimensional surface may be detected. This creates a finger controlled “mouse” computer input device. Motion of a finger along the surface of such sensors may allow a user to control the movement of an indicator on a display screen, and control a microprocessor device. Such techniques are particularly useful for small space constrained devices, such as cell phones, smart cards, music players, portable computers, personal digital accessories, and the like.

RELATED APPLICATIONS

This application is related to U.S. Non-Provisional application Ser. No.TBD, filed Dec. 14, 2007, entitled “Method and Apparatus for FingerprintImage Reconstruction,” and U.S. Non-Provisional application Ser. No.TBD, filed Dec. 14, 2007, entitled “Method and Algorithm for AccurateFinger Motion Tracking.”

This application is a continuation in part of, and claims the prioritybenefit of, U.S. Non-Provisional application Ser. No. 11/519,383, filedSep. 11, 2006, and also a continuation in part of U.S. Non-Provisionalapplication Ser. No. 11/519,362, filed Sep. 11, 2006.

This application is a continuation in part of, and claims the prioritybenefit of, U.S. Non-Provisional application Ser. No. 11/107,682, filedApr. 15, 2005. U.S. Non-Provisional patent application Ser. No.11/107,682 claimed the priority benefit of U.S. Provisional PatentApplication No. 60/563,139, filed Apr. 16, 2004.

This application is also a continuation in part of, and claims thepriority benefit of, U.S. Non-Provisional patent application Ser. No.11/112,338, filed Apr. 22, 2005. U.S. Non-Provisional patent applicationSer. No. 11/112,338 claimed the priority benefit of U.S. ProvisionalApplication 60/564,791, filed Apr. 23, 2004.

This application is also a continuation in part of, and claims thepriority benefit of, U.S. Non-Provisional patent application Ser. No.11/243,100, filed Oct. 4, 2005. U.S. Non-Provisional patent applicationSer. No. 11/243,100 claimed the priority benefit of U.S. ProvisionalPatent Application 60/615,718, filed Oct. 4, 2004.

BACKGROUND

The invention relates generally to technology for sensing and recordingfinger motion, fingerprints and, more particularly to systems, devicesand methods for finger motion tracking both alone, and in combinationwith fingerprint image processing and navigation operations.

Partial fingerprint scanners are becoming popular for a wide variety ofsecurity applications. In contrast to “all at once” fingerprintscanners, which capture an image of an entire fingerprint at the sametime, partial fingerprint sensing devices use a sensing area that issmaller than the fingerprint area to be imaged. By imaging only aportion of a fingerprint at any given time, the size and cost of apartial fingerprint sensor can be made considerably smaller and cheaperthan that of a full fingerprint sensor. However to capture a fullfingerprint image, the user must move his finger and “swipe” it acrossthe sensing zone of the partial finger print sensor.

Various types of partial fingerprint readers exist. Some work by opticalmeans, some by pressure sensor means, and others by capacitance sensingmeans or radiofrequency sensing means.

For example, one common configuration used for a fingerprint sensor is aone or two dimensional array of CCD (charge coupled devices) or C-MOScircuit sensor elements (pixels). These components are embedded in asensing surface to form a matrix of pressure sensing elements thatgenerate signals in response to pressure app lied to the surface by afinger. These signals are read by a processor and used to reconstructthe fingerprint of a user and to verify identification.

Other devices include one or two dimensional arrays of optical sensorsthat read light reflected off of a person's finger and onto an array ofoptical detectors. The reflected light is converted to a signal thatdefines the fingerprint of the finger analyzed and is used toreconstruct the fingerprint and to verify identification.

Many types of partial fingerprint scanners are comprised of linear (1dimensional) arrays of sensing elements (pixels). These one dimensionalsensors create a two dimensional image of a fingerprint through therelative motion of the finger pad relative to the sensor array.

One class of partial fingerprint sensors that are particularly usefulfor small device applications are deep finger penetrating radiofrequency (RF) based sensors. These are described in U.S. Pat. Nos.7,099,496; 7,146,024; and US Publication Nos. US 2005-0244038 A1, US2005-0244039 A1, US 2006-0083411 A1, and US 2007-0031011 A1, and thecontents of these patents and patent applications are incorporatedherein by reference. These types of sensors are commercially produced byValidity Sensors, Inc, San Jose Calif. This class of sensor mounts thesensing elements (usually arranged in a one dimensional array) on athin, flexible, and environmentally robust support, and the IC used todrive the sensor in a protected location some distance away from thesensing zone. Such sensors are particularly advantageous in applicationswhere small sensor size and sensor robustness are critical.

The Validity fingerprint sensors measure the intensity of electricfields conducted by finger ridges and valleys, such as deep fingerpenetrating radio frequency (RF) based sensing technology, and use thisinformation to sense and create the fingerprint image. These devicescreate sensing elements by creating a linear array composed of manyminiature excitation electrodes, spaced at a high density, such as adensity of approximately 500 electrodes per inch. The tips of theseelectrodes are separated from a single sensing electrode by a smallsensor gap. The electrodes are electrically excited in a progressivescan pattern and the ridges and valleys of a finger pad alter theelectrical properties (usually the capacitive properties) of theexcitation electrode-sensing electrode interaction, and this in turncreates a detectable electrical signal. The electrodes and sensors aremounted on thin flexible printed circuit support, and these electrodesand sensors are usually excited and the sensor read by an integratedcircuit chip (scanner chip, driver chip, scan IC) designed for thispurpose. The end result is to create a one dimensional “image” of theportion of the finger pad immediately over the electrode array andsensor junction.

As the finger surface is moved across the sensor, portions of thefingerprint are sensed and captured by the device's one dimensionalscanner, creating an array of one dimensional images indexed by order ofdata acquisition, and/or alternatively annotated with additional timeand/or finger pad location information. Circuitry, such as a computerprocessor or microprocessor, then creates a full two-dimensionalfingerprint image by creating a mosaic of these one dimensional partialfingerprint images.

Often the processor will then compare this recreated two dimensionalfull fingerprint, usually stored in working memory, with an authorizedfingerprint stored in a fingerprint recognition memory, and determine ifthere is a match or not. Software to fingerprint matching is disclosedin U.S. Pat. Nos. 7,020,591 and 7,194,392 by Wei et. al., and iscommercially available from sources such as Cogent systems, Inc., SouthPasadena, Calif.

If the scanned fingerprint matches the record of an authorized user, theprocessor then usually unlocks a secure area or computer system andallows the user access. This enables various types of sensitive areasand information (financial data, security codes, etc.), to be protectedfrom unauthorized users, yet still be easily accessible to authorizedusers.

The main drawback of partial fingerprint sensors is that in order toobtain a valid fingerprint scan, the user must swipe his or her fingeracross the sensor surface in a relatively uniform manner. Unfortunately,due to various human factors issues, this usually isn't possible. In thereal world, users will not swipe their fingers with a constant speed.Some will swipe more quickly than others, some may swipe at non-uniformspeeds, and some may stop partially through a scan, and then resume. Inorder to account for this type of variation, modern partial fingerprintsensors often incorporate finger position sensors to determine, relativeto the fingerprint sensor, how the overall finger position and speedvaries during a finger swipe.

One type of finger position indicator, represented by U.S. Pat. No.7,146,024, and US Publication Nos. US 2005-0244038 A1, US 2005-0244039A1 (the contents of which are incorporated herein by reference) detectsrelative finger position using a long array of electrical drive platesensors. These plates sense the bulk of a finger (rather than the finedetails of the fingerprint ridges), and thus sense the relative positionof the finger relative to the linear array used for fingerprint sensing.A second type of fingerprint position indicator, represented by U.S.patent publication US 2007-0031011 A1 (the contents of which areincorporated herein by reference), uses two linear partial fingerprintsensors, located about 400 microns apart. The two linear sensors use theslight timing differences that occur when a fingerprint swipe first hitsone sensor and then the other sensor to detect when a fingerprint edgepasses over the sensors. This technique can also detect relative speedof passage over the two partial sensors. This type of information can beused to deduce overall finger location during the course of afingerprint swipe.

Another device is described in U.S. Pat. No. 6,002,815 of Immega, et al.The technique used by the Immega device is based on the amount of timerequired for the finger to travel a fixed distance between two parallelimage lines that are oriented perpendicular to the axis of motion.

Still another technique is described in U.S. Pat. No. 6,289,114 ofMainguet. A device utilizing this method reconstructs fingerprints basedon sensing and recording images taken of rectangular slices of thefingerprint and piecing them together using an overlapping mosaicalgorithm.

In either case, once finger position is known, each of theone-dimensional partial fingerprint images can then be annotated withadditional (and optional) time data (time stamp) or finger (finger tip,finger pad, fingerprint location) location data (location stamp). Thisoptional annotation information, which supplements the “order of dataacquisition” that would normally be used to keep track of the multiplestored partial fingerprint images in memory, can be used to help tocorrect distortions (artifacts) when the various one dimensional partialimages are assembled into a full two dimensional fingerprint image.

For example, if the user momentarily stops moving the finger during thefinger swipe, the system will generate a series of nearly identicalpartial (one dimensional) fingerprint images. These images will havedifferent orders of acquisition, and differing time stamps, which couldconfuse a processor when it attempts to create a correct two dimensionalfull fingerprint image. However if the fingerprint scanner also has afinger position sensor, the finger location data stamp associated withthese nearly identical one dimensional partial fingerprint images willprovide evidence that the finger stopped because the finger locationdata linked to these various one-dimensional partial fingerprint imageswill be almost the same. The computer processor that reassembles thepartial fingerprint images into the complete fingerprint image can beinstructed or programmed to also analyze the finger position (location)data, and perform appropriate image corrections when the location datashows that the finger paused during a scan.

U.S. Pat. Nos. 7,099,496, 7,146,024, and US Publication Nos. US20050244038 and US 20050244039 describe a combination fingerprint sensorand finger location apparatus, and the contents of these application areincluded herein by reference. FIGS. 11-18 of U.S. Pat. No. 7,099,496show various ways in which finger location sensors and partialfingerprint images can be packaged together to produce a system capableof reproducing an entire fingerprint. FIG. 11 of U.S. Pat. No.7,099,496, shows a fingerprint sensor that contains both a fingerprintimager (110), (114), (116), (118) and fingerprint location pickup plates(112), and (1120) through (1162).

Similarly application US Publication No. US 2005-0244038 A1, FIG. 1shows an overview of how a finger position sensor (112) and a partialfingerprint sensor (swiped image sensor) (110) can be integrated into anoverall fingerprint scanner, and FIGS. 5 through 11 and 13 through 16show how a finger moves over a series of sensor plates in the fingerposition sensor in the course of a finger print swipe.

One drawback of these earlier approaches was that the system was stilltoo sensitive to individual variations in user technique. One way toaddress this issue, discussed in application US Publication US20050244039A1, is to assist the user to produce a finger swipe that thesystem can properly read by giving the user visual or audio feedback asto if the finger swipe produced finger location and speed data that wasreadable by the system. This way, the user can be encouraged optimizehis or her finger swipe technique. However alternative approaches, suchas improved signal analysis techniques that make the system moretolerant to variations in user technique, either with or without audioor visual user feedback are also desirable.

One of the reasons why these earlier approaches were overly sensitive tovariations in use technique is that the systems were not accurateenough. In a rate based sensor, noise can occur when the sensor that isconfigured to detect the finger and take a reading of finger features isunable to accurately detect the location and motion of the finger whileit is being swiped. The result can be noise, such as electronic noise,resulting from a transition of the finger from one sensor element toanother. As a result, the finger motion rate calculation is not totallyaccurate because the signal noise occurring from one sensor element toanother creates uncertainty with respect to the location and timing atwhich a finger transitions from one sensor to another.

Therefore, there exists a need in the art to more accurately sensefinger swiping motion across a fingerprint sensor and to accuratelycalculate the speed and location of the finger that is in motion acrossthe sensor. There also exists a great need in the art for a moreefficient means to accurately sense and capture fingerprints on portablemicroprocessor controlled devices (e.g. cell phones, smart cards, PDA's,laptop computers, MP3 players, and the like). There is also a need formore convenient and efficient ways to provide navigation and controloperations on such portable devices. As will be seen, the inventionprovides for these multiple needs in an elegant manner.

BRIEF SUMMARY OF THE INVENTION

Further improvements in the finger location and movement sensingtechnology previously disclosed in U.S. Pat. Nos. 7,099,496 and7,146,024, and also Publication Nos. US 20050244038 and US 20050244039that are commonly assigned to applicant and incorporated herein byreference, are possible, and some of these additional improvements aredescribed herein. In particular, the present invention teaches improvedsignal analysis techniques that can enable finger locations and movementduring a finger swipe for a fingerprint scan to be determined withimproved accuracy and precision, greater resistance to noise (falsesignals), and over a wider variety of different finger swipe techniques.The present invention also teaches techniques to use this improvedinformation to produce more accurate fingerprint images.

The motion of a finger as it is being swiped across various sensorelements or plates generates noise as the finger transitions from onesensing plate to another plate. Finger motion may be detected byanalyzing the sensor signals as a function of time, detecting stableregions of the signals, followed by regions where the sensor signalbecomes distorted or noisy. This will usually define finger motion fromone sensor element to the other. The problem, however, is that the noisyarea is both broad and poorly defined, and this leads to inaccurateresults. This is particularly true when the finger being swiped acrossthe sensor changes in velocity or direction. This commonly happens withdifferent types of users, each of whom may have their own uniquetechniques. Alternative algorithms that determine finger position andmovement with higher accuracy are disclosed, and these algorithms inturn allow finger motion sensors to be produced that can track smallvariations in finger position and velocity with higher degrees ofaccuracy.

These enhanced accuracy finger position and motion sensors can be usedin a greater variety of different applications. In one type ofapplication, these higher accuracy finger motion sensors can be used inconjunction with partial fingerprint scanners to produce improvedfingerprint imagers that function with greater robustness and accuracy.In a second type of application, these higher accuracy finger motionsensors may be used (either with or without a partial fingerprintimager) to control electronic devices. When several of these fingermotion and position sensors are aligned in different directions, fingermotion over a two dimensional surface may be detected. This allows a newtype of finger controlled “mouse” computer input device to be created.Motion of a finger along the surface of such sensors may allow a user tocontrol the movement of an indicator on a display screen, and control amicroprocessor device. Such techniques are particularly useful for smallspace constrained devices, such as cell phones, smart cards, musicplayers, portable computers, personal digital accessories, and the like.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a diagrammatic view of a multiple detection element (plate)finger motion sensor.

FIG. 1B shows a close up of a finger surface as it contacts variousdetection elements (plates) of the finger motion sensor.

FIG. 1C shows the approximate dimensions of the individual sensingelements (plates) of a finger motion sensor array.

FIG. 1D shows the motion of a finger over the finger motion sensorarray.

FIG. 1E shows a flow chart of the algorithm used to determine fingermotion.

FIG. 1F shows a detail of the timing of the signals that are produced bythe finger motion sensor array.

FIG. 1G shows a flow chart of the improved algorithm used to determinefinger motion.

FIG. 1H shows how finger motion data may be combined with a series ofpartial fingerprint images, and used to in the process of reassemblingthe partial fingerprint images into a complete image of a fingerprint.

FIG. 1I shows a diagram of a deep finger penetrating radio frequency(RF) based partial fingerprint sensor equipped with two finger motionsensing arrays.

FIG. 1J shows how the finger position data may be used to helpreconstruct a fingerprint image from overlapping partial fingerprintimages.

FIG. 1K shows how the finger position data may be used to helpreconstruct a fingerprint image from non-overlapping partial fingerprintimages that may contain one or more gaps between the partial images.

FIG. 1L shows how the finger position data may be used to helpreconstruct a fingerprint image from mixed partial fingerprint images,some of which are overlapping, and some of which contain one or moregaps between the partial images.

FIG. 1M shows how the finger position data may be used to helpreconstruct a fingerprint image from partial fingerprint images thatsuffer from a high degree of redundancy and overlap between images.

FIG. 2A shows a combination partial fingerprint scanner and fingermotion detector device that has with multiple finger motion sensorarrays arranged at various angles.

FIG. 2B shows the motion of a finger over the device shown in FIG. 2A.

FIG. 2C shows the combination partial fingerprint scanner and fingermotion detector device shown in 2A embedded as a component of a smallportable electronic device containing a display screen. Here the motionof a finger over the detector device may control the movement of acursor on the screen.

FIG. 2D shows that integrated circuit chip used to drive the fingermotion detector arrays may be mounted on a flexible circuit or film somedistance away from the actual finger motion detector arrays.

FIG. 2E shows an example of the advantages of capturing finger motion intwo degrees fingerprint detectors. Here a complete fingerprint isrecreated by assembling partial fingerprint images obtained by movingthe finger in a vertical direction.

FIG. 2F shows an example of the advantages of capturing finger motion intwo degrees fingerprint detectors. Here a complete fingerprint isrecreated by assembling partial fingerprint images obtained by movingthe finger in a horizontal direction.

FIG. 3 shows a view of a finger moving over a combination finger motionsensor and partial fingerprint imager.

FIG. 4 shows a view of a finger moving over a combination finger motionsensor and partial fingerprint imager. Here the finger motion sensor isequipped to sense finger motion in two dimensions.

FIG. 5A shows a view of a finger moving over an alternate combinationfinger motion sensor and partial fingerprint imager. Here the fingermotion sensor is equipped to sense finger motion in two dimensions.

FIG. 5B shows a view of a finger moving over another alternatecombination finger motion sensor and partial fingerprint imager. Herethe finger motion sensor is equipped to sense finger motion in twodimensions.

FIG. 6 shows a view of a finger moving over an alternate combinationfinger motion sensor and partial fingerprint imager.

FIG. 7 shows a view of a finger moving over another alternatecombination finger motion sensor and partial fingerprint imager. Herethe finger motion sensor is equipped to sense finger motion in twodimensions.

FIG. 8A shows that the integrated circuit chip used to drive the fingermotion detector arrays may be mounted on a flexible circuit or film somedistance away from the actual finger motion detector arrays. Here thefinger motion detectors are configured to detect finger motion in twodimensions.

FIG. 8B shows that the integrated circuit chip used to drive the fingermotion detector arrays may be mounted on a flexible circuit or film somedistance away from the actual finger motion detector arrays. Here thefinger motion detectors are configured in an alternate configuration todetect finger motion in two dimensions.

FIG. 8C shows that the integrated circuit chip used to drive the fingermotion detector arrays may be mounted on a flexible circuit or film somedistance away from the actual finger motion detector arrays. Here thefinger motion detectors are configured in an alternate configuration todetect finger motion in two dimensions.

FIG. 9 shows an example of some of the circuitry used to read a fingermotion sensor.

FIG. 10 shows an example of an algorithm used to transform finger motiondata into navigational commands for a computerized device.

FIG. 11 shows an example of an algorithm used to transform finger motiondata into cursor motion commands for a computerized device with adisplay screen.

DETAILED DESCRIPTION

The techniques discussed here can generally be used with the sensingcircuits previously described in U.S. Pat. Nos. 7,099,496 and 7,146,024,and also U.S. Publication Nos. US 2005-0244038A1 and US 2005-0244039A1that are commonly assigned to applicant and incorporated herein byreference. Please see these applications for a more detailed discussionof the electronic elements. The present invention is focused on signalanalysis techniques, methods, and algorithms, and improved fingerprintsensors and navigational devices that use these previously disclosedfinger position sensing devices. Thus the present application will notreiterate the details of these previously discussed electrical circuitsunless they are relevant to the present invention.

Referring to FIG. 1A, a side diagrammatic view of a finger positionsensor in contact with a finger is illustrated according to theinvention. The sensing array (sensing array medium) (100) is illustratedin somewhat of a side diagrammatic cut-away view that illustrates theindividual sensing plates P_(n), P_(n-1), P_(n-2), . . . P₃, P₂, P₁ andP₀.

In operation by a user, the finger (104) is swiped against a top surface(105) of the sensing array, where the finger surface (106) is physicallyjuxtaposed against surface of the individual plates, P_(n) through P₀and a the location and movement of the bulk of the finger is captured bythe sensing array. Normally, a partial fingerprint scanner that capturesa more detailed image of a portion of the fingerprint will also bepresent (see, for example, present FIG. 1I, FIGS. 3 and 4, and FIGS. 5-8of Publication US2005-0244038A1), but for simplicity, this is not shownin this figure.

The sensing array has a combination of individual sensing elements thatare used for sensing the presence and location of the bulk of the fingersurface (106) when it is in contact with the surface (105) of thesensing array. These individual elements can be used to sense thepresence, location, and motion of the bulk of the finger with respect tothe sensing array surface (105). This information can be used todetermine the presence and motion of the finger skin (which holds thefingerprint) (106) with respect to sensing array surface (105), but donot capture an image of the fingerprint itself. This is done by thepartial fingerprint imaging sensors (not shown).

This bulk finger motion and location information, together with partialfingerprint images obtained by the fingerprint imager, is used toreconstruct a captured image of the fingerprint surface on the undersideof the finger (106) for use in authorizing a user by identifying afingerprint with a stored version of the user's fingerprint.

Thus when finger (104) is moved in the swipe direction (A), the sensorelements (100) can detect the finger's presence, velocity, acceleration,and direction. Depending upon the particular application, thisinformation can be used in different ways. In some applications, onlyfinger velocity information may be desired. In other applications,finger acceleration and direction may also be desired in order to allowa more complete image of the total fingerprint to be reconstructed froma mosaic of partial fingerprint images. Here, speed, acceleration anddirection information can help correct for distortions that may occur ifthe finger is moved in an abnormal or unpredictable direction.

In another application, this information can be used for navigationalapplications, can and be used somewhat like a computer “mouse” to helpthe user command a microprocessor equipped computerized device.

Typically the sensing array (100) is connected to a sensor circuit(116), often by link (117) such as a wire connection or flexible circuitboard. Using a sensor circuit (116) that is physically separated fromthe array (100) allows the sensor circuitry, which may be delicate orprone to picking up environmental noise, to be placed in a moreprotected or convenient location, and can help improve systemrobustness.

The sensor circuit (116) may include a processor (116 a) and alsopersistent memory (116 b) for storing general system process softwareand instructions executable by the processor. The sensor circuit mayalso include memory that can be read and written to such as RAM (116 c).Motion process software (116 d) may be included in the RAM or persistentmemory (116 b) (typically this software will require at least some RAMto function).

The motion process software (116 d), when executed by the processor (116a), can activate the sensing array and interpret signals from the arrayvia I/O (116 e). This allows the processor to receive data from thesensing array as a fingerprint surface (106) is swiped across thesurface (105). As previously discussed, such data may include fingerpresence, location, speed, acceleration, and direction. As previouslydiscussed, this data can be used to determine the finger location wherea partial fingerprint sensor (not shown) has received a partialfingerprint image, and using this data, a series of partial fingerprintimages can be reassembled into a mosaic representative of a completefingerprint image. In this diagram, the partial fingerprint imager (notshown) is located proximal to the sensing array medium (100).

Here, improved motion process software is described that enables fingerlocation and speed to be determined with superior accuracy and superiorresistance to variations in user technique. In order to describe howthis improved motion process software operates, the system will bedescribed in further detail.

In order to properly analyze the signal generated by the variousindividual sensing plates P_(n), P_(n-1), P_(n-2), . . . P₃, P₂, P₁ andP₀ as the finger moves across these plates, a significant amount ofsignal analysis must be done. In particular, the data from the variousplates must be properly interpreted, and then properly meshed with thedata from a partial fingerprint sensor in order to generate a usableresult.

FIG. 1B shows a close up close up side view of a finger being swipedacross the individual sensing elements of the sensing array. Thefingerprint surface (106) is swiped in direction (A) across theindividual sensing elements P_(n-1), P_(n-2) and P_(n-3). There are gapsbetween the different sensing elements (103), and the fingerprintsurface (106) moves from one sensing element to another.

In order to determine the location of the fingertip (107), the processorneeds to receive a signal that indicates a finger transfer from onesensor plate to another. Although the analog signal data from thevarious individual sensing elements could be rounded off or truncated toproduce simple binary “on plate” or “off plate” signals (that is, is thefinger “on” or “off” sensing element P_(n)), as was done with someprevious implementations, such a low resolution approach discardsimportant information. Much useful information can also be found byanalyzing slight variations in the plate signals (that is, digitizingthe analog signal generated by the plates to a higher degree ofprecision), and then using this information, together with time ofacquisition data and sensor plate location data, to determine fingerposition and movement more precisely. Here, techniques to do this moreprecise analysis are described.

Referring to FIG. 1C, a more detailed view of the individual sensingelements is illustrated. Each element has a fixed width, “w”, and eachelement has a distance from an adjacent element, “d”. From the centers,“c”, of each sensing element to an adjacent element is a center length“L”, which is the distance from one center to the center of an adjacentelement. As illustrated, element P_(n) has a width, “w₁”, and a center,“c₁”. The distance between this individual sensing element and the nextsensing element P_(n-1) is “d₁”. The next sensing element P_(n-1) has awidth “w₂” and a center of “c₂”. The center point length between the twoelements is “L₁”.

In essence then, the signal analysis problem is one of using theelectrical signals generated by the various sensing plates to determinethe speed of motion of the fingerprint surface, and its location, withrespect to the surfaces s₁, S₂ . . . s_(n) of the respective sensingelements. By carefully analyzing the intensity and timing of theseelectrical signals, and using the techniques taught herein, the positionand movement of the finger may be determined with improved accuracy.

In order to facilitate this discussion, the dimensions of the sensorarray will be defined. Here, the fingerprint surface transfers from onesensor element to another between the gaps “d”, the finger surface movesbetween each and every plate from plates P_(n) through P₀, and theseplates have a width w_(n), a surface s_(n) and a center point c_(n).

Usually, a sensor array will contain a large number of individualplates, and these are shown in FIG. 1 d. Here again, finger (104) isswiped across sensor array (102) in swipe direction (A), and the changesin the electrical characteristics of the plates at various times aredetected by sensor circuit (116). Usually the sensor circuit (116) willrapidly scan across the various plates Pn to P0, and report a series ofelectrical values from each plate for each scan and, relevant to thisdiscussion, for each scan time. The result from one of these scans isshown as the reading measure (119).

Referring back to FIG. 1B for a moment, note that as the edge of thefinger (107) passess over plate P_(n), there is a region of partialfinger contact. That is, while the finger is completely over plateP_(n-2), the finger edge (107) is only partially over plate P_(n). Thisregion of partial contact can be detected by the electrical measurementsof the plate, and the relative degree of finger edge contact (107)influences the electrical signal returned by plate P_(n).

As a finger tip (107) passes over a particular plate, the plategenerates varying electrical signals. In particular, when the finger tip(107) first contacts either edge of the plate, for a while anintermittent (noisy) electrical signal is generated. Then, when the edgeof the finger migrates away from the edge more towards the middle of theplate, there is a region where the electrical signal is more stable.Finally, when the edge of the finger migrates toward the opposite edgeof the plate, there is another region where again a noisy electricalsignal is generated. Thus the timing of the regions of noise andstability contains useful information relative to the precise locationand speed of the bulk of the finger, which the present inventionexploits to improve the robustness and accuracy of the fingerprintscanner.

Thus, according to the invention, the sensor circuit is able todetermine a stable period where it is determined that the fingerprintsurface is present and being sensed by a particular sensing element andthe readings are continuous. The processor (116) and motion processsoftware (116 d) can use this information, in conjunction withpreprogrammed knowledge of the geometry and dimensions of the varioussensor plates (which can be stored in persistent memory (116 b) oralternatively in RAM (116 c)), to determine, with improved accuracy,exactly what portion of the finger surface containing fingerprint (106)are passing over a fingerprint imager (sensor) at any given instant oftime. This improved finger location information allows the variouspartial fingerprint images, produced by the fingerprint imager (sensor),to be stitched together by software into a more accurate completefingerprint image. The complete fingerprint image is more accuratebecause the relative location of the various partial fingerprint imagesis better defined. An additional advantage is that by this more detailedanalysis of noise and stable periods, more marginal finger swipes can be“saved”, and users will be able to use the system with less training, oralternatively will not need to repeat finger swipes so often. The endresult is improved user satisfaction and higher commercial success.

In order to do this, the motion process software (116 d) will performalgorithms or processes similar to that shown in FIG. 1E. The processstarts in Step (202). In Step (204) the fingerprint is sensed by thesystem. From here, there are two separate sub processes that occurnearly simultaneously. In Step (206), the system determines motiontiming throughout the sensing period between stable periods of thesensor signal. This process simply calculates the speed and location ofthe bulk of the finger as it is being swiped across the sensor. In Step(208), the speeds of bulk finger motion are calculated (and also bulkfinger location), based upon the system's knowledge of the dimensionsand locations of the various sensing plates.

In the parallel process (210), the fingerprint images are capturedthroughout the sensing period by the fingerprint sensor. In thisseparate process, which can be completed essentially simultaneously withthe bulk finger speed and location sensing, the actual image of thefingerprint (formed from the underside of the bulk finger) that is beingswiped across the sensor is being recorded.

It should be appreciated that the accuracy and success of this approachis dependent on the accuracy of the finger location and movement data.Earlier approaches, which neglected finger edge and noiseconsiderations, were essentially blind to finger locations while thefinger traversed a given sensor plate P_(n). By contrast, by making useof the fact that the plate signal becomes nosier as the fingerapproaches plate edges, and becomes more stable while the finger is nearthe center of the plate (process step (206)), additional information isavailable to help determine finger location and movement to higherprecision.

Put another way, according to the invention, the process step (206) isan improved process step which more accurately calculates the bulkfinger motion and relative location as it moves from one sensor plate(element) to another during a finger swipe. This data, in combinationwith speed and location calculations (208) determines the location ofthe fingerprint images (210) on the finger surface more precisely. InStep (212), the fingerprint image is reconstructed using the imagescaptured in Step (210), and also the speed calculations calculated inStep (208). Using this information, the fingerprint image can bereconstructed more accurately. Usually this reconstructed fingerprintwill then be compared to an authorized fingerprint in order to givesecurity authorization for the user.

FIG. 1F shows a simplified view of the sensor data that is obtained fromthe finger position sensor array during a swipe. The Signal (200) is anidealized example of a signal that would typically be produced by thesensor elements as during a swipe. The fingerprint signal begins in Step(202) when a fingerprint first is set upon the fingerprint sensor.Typically, a noisy signal (202) initially occurs because the location ofthe fingerprint with respect to the separate elements of the fingerprintsensor is initially only partially defined as the finger settles ontothe sensor plates. As the finger moves from one plate to the next, theoverall signal changes, and this is represented by the stair steppattern in FIG. 1F.

Once the finger has settled on to the sensor, the signal will then entera region of relative stability (203A). However this stability thenbecomes interrupted as the finger then begins to move across the varioussensor plates during the progress of the swipe. During the swipe, as thefinger tip (107) transitions from between one plate gap and the other,the signal changes as a series of somewhat noisy steps.

The transition stage as the finger tip moves off of a first sensingplate (usually the plate furthest away from the fingerprint imagingsensor) is shown in the area circled by region (204). As the fingermoves, the originally steady signal once again becomes unstable andunsure (to keep the diagram simple, this signal noise is not shown).During transition region (204) noise in the signal makes it difficult toprecisely determine if the finger is on or off that particular plate,and approaches, which attempt to simplify the signal into a binary “onplate” or “off plate” result are thus prone to error. In reality, thesignal transition region where the finger tip transitions from one plateto another may be relatively broad, beginning on the left side of theregion (204L) and ranging to (204R), with a center point (204C).

One possible way to cope with the problem is to look at the timeinterval where the noise begins (passess a certain threshold), look atthe time interval where it ends (again passes a certain threshold), anduse the midway point to localize the exact time when the tip of thefinger passed a given plate gap. Although this works, the results arestill prone to error because the beginning time and ending time of anoisy signal can be rather indefinite. Sometimes random noise will causethe beginning tune of the noisy signal to be immediately detected, andsometimes, due to random fluctuations, the beginning time of the noisysignal will not be immediately detected. Similarly sometimes randomnoise will cause the end time of the noisy signal to be immediatelydetected, and sometimes the end time of the noise signal will not beimmediately detected. Thus the “midway noise time” approach to preciselylocate finger location and velocity does not produce timing results withoptimal accuracy, and can produce many small timing errors. These timingerrors in turn translate into bulk finger velocity and location datawith suboptimal accuracy. This in turn leads to suboptimal accuracy inassembling a complete fingerprint scan from a mosaic of partialfingerprint images.

In a more favored embodiment of the invention, the transition is notdetermined by the center of the noise field (204C). Rather it isdetermined by the center point of the stable (noise free) region(203(b)) and the center point of a subsequent stable (noise free) region(203(c)).

Returning to FIG. 1B for a moment, on a physical level, this change inalgorithm is somewhat like trying to define when the fingertip (107) ismidway over a plate, rather than trying to determine when the fingertip(107) is midway over a gap (103). Whereas small fluctuations in fingerposition will have a big impact on the finger tips being over or notover a gap (103), and thus have a big impact on the noise produced bysensing plates P_(n) and P_(n-2), small fluctuations in finger positionwhen it is midway over a plate (in FIG. 1B, finger tip (107) is midwayover plate P_(n)) will have a relatively small impact on the signalproduced by plate P_(n). Thus looking at the position when the fingertipis midway over a plate, rather than looking at the position when thefingertip is midway over a gap (103), is less sensitive to randomfluctuations, and can produce results with higher accuracy.

Returning to FIG. 1F, the center point of the stable (noise free)regions is shown in (203(a)-(h)). As previously discussed, use of thesenoise free regions is favored because it produces more accurate results.First looking at the timing t_(range-1), it is easier for a system tocalculate the beginning and the end of that range. Once the center pointis known, the center point to subsequent center point times can be usedto determine the location of the finger as it moves across the sensorduring a particular period of time. Thus, using the timing one centerpoint of a stable region, such as (203 b) to the center point of asubsequent stable region (203 c), a more accurate timing t_(n) can becalculated from one point to the next. Using this process, the differenttimes that the bulk of the finger passes different sensor plates can bedetermined as timing intervals t_(n), t_(n-1), t_(n-2) . . . t₃, t₂, t₁and t₀.

Processor (116 a) and motion process software (116 d) can use thisinformation to deduce finger location and speed. A flow chart for doingthis is shown in FIG. 1G. The process starts in Step (302). In Step(304), the fingerprint sensor (or the finger position sensing array) ismonitored for the presence of a finger being swiped across it. In Step(306), it is determined whether or not the presence of a finger has beendetected. If not, the process returns to Step (304) where monitoring forpresence continues. Here for example, if there is an absence of signalfrom the sensor, the circuit is assumed to be dormant or inoperable.

After the finger presence is sensed, then the system is initiated andthe sensing of a finger location and fingerprint scan commences. In Step(308), a sensor circuit connected to the finger position sensing array(see FIG. 1, sensor circuit (116)) is configured to monitor the signalfor time periods of stability between noise, as previously described inFIG. 1F. As previously discussed, when the finger is first applied tothe sensing array, the signal will initially suffer from noise. Thus thesystem next monitors this initial noise to dissipate before initiatingthe speed sensing. The signal stability is monitored in step (308), andenters a hold loop until a stable signal is detected (310). Once thesignal is stable, the process goes from Step (310) to (312) and sets aninitial timing value for the stable period, t_(begin). The process thenenters another loop where it continues to monitor stability of thissignal in Step (314). (As can be seen in FIG. 1F, this corresponds tofirst waiting out unstable period (202), and looking for the start ofthe stable region (203(b)).

Once the signal looses stability, this is detected in Step (316), andthe ending time t_(end) of that formerly stable period is assigned inStep (318). In Step (320), the stability time period is calculated bythe following formula:t _(stable)=(t _(end) −t _(begin))/2,

In Step (322), the value Of t_(stable) is stored. In Step (324), it isdetermined whether or not the sensing processes are completed.

If the noisy signal lasts for less than a preset time, such as less than2 milliseconds, the system may determine that the finger was verybriefly swiped but now is no longer present on the sensor. In this case,the process may return to Step (308) and continue to monitor for astable signal. If the process is completed, then the process proceeds toStep (326) where the end of the sensing period is determined and theentire fingerprint is presumed to swiped and sensed. The process thenreturns to Step (304) where the system continues to monitor for presenceof another fingerprint. As a result of this process, the finger positionsensing array, processor, and software thus act to collects the datapreviously discussed in FIG. 1F.

As previously discussed in FIG. 1E, while the finger location and motiondata is being collected by the finger sensing array, a fingerprintimager or scanner will be collecting (210) a (usually large) number ofimages (usually 1 dimensional fingerprint scans) of various portions ofthe fingerprint that is located on the underside of the finger. In orderto assemble a complete fingerprint, the location and speed data of thefinger from which each partial fingerprint image was obtained (raw datapreviously processed in FIG. 1F) is used to essentially assemble amosaic of partial fingerprint images, producing the full fingerprint.This process is shown in FIG. 1H.

In FIG. 1H, a process for reconstructing a fingerprint using the valuesgenerated in the process from FIG. 1G is illustrated. First, the systemmay establish reconstruction parameters—usually the various coefficientsand criteria used to create the final fingerprint mosaic. In Figure(406), the timing data calculated in the timing process discussed inFIG. 1G above is retrieved, retrieving t_(end) and t_(begin). Then, thespeed is calculated from point to point over a period in Step (408). InStep (410), the partial fingerprint images are retrieved.

The fingerprint image reconstruction strategy will differ somewhatdepending upon how fast the finger was swiped. Usually the partialfingerprint images will be collected at a set number of images persecond, producing a defined number of different partial fingerprintimages every second. If the finger was swiped slowly, then so much dataand so many partial fingerprint images may have been collected as tocreate much redundancy, and many of these redundant partial fingerprintimages can and should be discarded so as not to clutter system memoryand processing resources with a lot of redundant data. On the otherhand, if the finger swipe was fast, the reverse problem may haveoccurred, in which an insufficient number of partial fingerprint imagesmay have been recorded. In this case, since fingerprints usually consistof a series of connected grooves, it is usually adequate to make up forat least a small amount of missing data by interpolating betweenneighboring images (image segments).

In Step (412), the finger speed data from the fingerprint sensing arrayis used to determine if the fingerprint motion was fast enough to betreated by the “fast swipe” algorithm. If the swipe was a fast swipe,then any gaps between the various partial fingerprint images (againusually one-dimensional fingerprint scans), here called “image segments”may be filled in by interpolation Step (414). If the swipe was a veryslow swipe, then redundant (overlapping) partial fingerprint images maybe discarded Step (420), and the selected partial fingerprint images(image segments) are used to reconstruct the entire fingerprint. If theswipe was neither too fast or too slow (422), then most of the imagescan be used. In either of three scenarios, the process ends up in Step(416) to determine whether or not the last image has been used in orderto reconstruct the fingerprint image. If it is not the last image, thenthe process returns to Step (406) where the timing data is retrieved.Once the last image is captured, as determined in Step (416), then thereconstruction of the image is ended in Step (424).

FIG. 1I shows a diagram of another embodiment of the present invention.In this embodiment, a deep finger penetrating radio frequency (RF) basedpartial fingerprint sensor (configured generally according to theteaching of U.S. Pat. No. 7,146,024, and US Publication Nos. US2005-0244038 A1, US 2005-0244039 A1) is equipped with two finger motiondetection sensor arrays (510), (514), each mounted at an angle to theother, and one partial fingerprint imager sensor (530). In thisconfiguration, the motion of finger (501) is detected by both motiondetection sensor arrays (510) and (514) and the underside of the fingerpasses over partial fingerprint scanner (530). These sensors can all becontrolled by the same motion sensor and fingerprint sensor drivercircuit (540) located on integrated circuit chip (542). Data from motionsensing array (510) passes to circuit (540) via circuit traces (512)(for simplicity, only one trace is shown but in actuality, multipletraces may be used). Data from motion sensing array (514) passes tocircuit (540) via circuit traces (516) (again for simplicity, only onetrace is shown but in actuality, multiple traces may be used). Partialfingerprint imager (530) has multiple excitation lines (532) whichproduce electrical signals that are influenced by the depth offingerprint ridges, and these signals are picked up by electrode (534)and relayed back to circuit (540). Integrated circuit chip (542) mayalso have an optional microprocessor core (544) and optional onboardvolatile or non-volatile memory (546). This processor (544) mayimplement one or more motion and image sensing algorithms, and in someembodiments may be considered to be processor (116 a) FIG. 1A. Similarlymemory (546) in some embodiments may be considered to be memory (116 b)or (116 c) or (116 d) from FIG. 1A. The results of the scan are outputvia output line (548). Processor (544) may also be controlled by input(550) from circuits outside of chip (542).

In operation, the motion of finger (501) will generate differentialmotion signals on motion sensor arrays (510) and (514). In theconfiguration as shown, if the two arrays produce an identical motionsignal, then the finger is proceeding directly towards the partialfingerprint scanner (530). If motion sensor (510) is producing moremotion, then the finger is veering to the left. If motion sensor (514)is producing more motion, then the finger is veering to the right. Inthis way, the same device may produce both fingerprint scans, and mayalso produce two dimensional finger motion information that can be usedin a “mouse like” manner to control a cursor or perform other controloperations on an electronic device. Also in this configuration, sensingarrays (510), (514), (530) and traces (512), (516), (532), (534) may allbe mounted on the same flexible film-like circuit, and IC chip (542) maybe also mounted on this flexible film-like circuit a few centimetersaway in a more remote location that is less prone to be subjected toenvironmental stress. In some embodiments, this configuration mayresemble FIG. (2D) or (8A).

Some examples of how a collection of partial fingerprint images arereassembled to form complete fingerprint images are shown in FIGS. 1J to1M.

In FIG. 1J, the various partial fingerprint images I₁, I₂, I₃, I₄, I₅and I₆, (here termed “image blocks” or “segments”) have partial overlap.These sections can be used to reconstruct the image, and the overlappingimage data can be excluded. This type of situation is associated withslow fingerprint scans.

In order to determine which partial fingerprint images should bediscarded or excluded, it will sometimes be useful to perform imageanalysis to confirm or detect image redundancy. Often, this can be doneby simple algorithms, such as, on a per pixel basis, subtracting a firstimage from a second image, and then summing up, over the pixels in theimage, the difference and determining if this difference is less than apreset threshold. If so, then the images are likely to be redundant.More sophisticated redundancy determination algorithms may also be used.

By contrast, in FIG. 1 k, the problem of partial fingerprint image gapscaused by too rapid a swipe is illustrated. As can be seen, there aregaps G₁, G₂, that illustrate the gaps between the images captured, I₁,I₂, and I₃. In practice, these gaps would need to either be corrected byinterpolation at to properly reconstruct the fingerprint image, and ifthe number of gaps exceeds some preset criteria, the data should bediscarded as being invalid, and the user invited to repeat thefingerprint scan. Again, like FIG. 1 j, these image sections, I₁, I₂,and I₃ are large in comparison to those in practice (the figure showstwo dimensional images, whereas typically the partial fingerprint imagesare actually one dimensional scans), but are shown here forillustration.

Various types of interpolation may be used to correct for the missingdata between images. The interpolation may be an adaptive algorithm thatchanges its parameters depending upon the image data, or a non-adaptivealgorithm that treats all pixels in the image the same way. If a nonadaptive algorithm is used, this algorithm may use a variety ofdifferent techniques including bicubic, bilinear, lanczos, nearestneighbor, sinc, spline or other methods. If an adaptive method is used,this method may use a variety of different techniques including GenuineFractals, PhotoZoom Pro, and Qimage. Often, it will be convenient to usenearest neighbor interpolation methods or bicubic interpolation methodsbecause such methods tend to require less microprocessor processingtime. Anti-aliasing techniques to remove the jagged edges that separatethe images on one side of a gap from the image on the other side of thegap may also be used.

The problem of an inconsistent fingerprint swipe, with mixed slow andfast regions, is shown in FIG. 1L. Here partial fingerprint images, forexample, I₁ and I₂ are separated by gap g₁, indicative of a fast swipeover this portion of the fingerprint as it is swiped across the sensor.Also, image I₂ and I₃ also have gap g₂, showing that the swipe was fasthere as well. However image sections I₃ and I₄ are overlapping,indicating that the finger slowed down to normal speed over this portionof the swipe. Here, the missing information in gap g₁ and g₂ can becompensated for by interpolation, but no such step is needed for imagesI₃ and I₄.

The problem of a very slow fingerprint swipe is shown in FIG. 1M. Heremany of the partial fingerprint images I₁ through I₇, are completelyoverlapping and redundant. For example, between image I₁ and I₃, thesetwo images are slightly overlapping, yet capturing different portions ofthe fingerprint. They can both be used in reconstructing the fingerprintimage. By contrast, image I₂ overlaps both image sections I₁ and I₃,making this image capture redundant. The system can thus omit thisparticular image in order to save processing time. Similarly, betweenimages I₄ and I₆, image I₅ is completely overlapping with images I₄ andI₆, making the image I₅ also redundant. Thus image I₅ can also beomitted. This saves memory and processing time and power.

Alternate Embodiments

Alternate embodiments and sensor array configurations are also possible.In operation of another embodiment of the invention, the linear sensorarray senses and captures fingerprint features in the form of a stringof data signals by first sensing the features in an initial sensing andcapture, and this is followed by one or more subsequent operations wherea sample is taken of a subset of the fingerprint features are capturedagain over a known time period. This time period may be predetermined ormeasured as time progresses between sensing and capturing of thesamples. Once at least two samples are taken, a subsequent sample iscompared against a previous sample to determine the amount shift of theprevious sample relative to the subsequent sample. In one embodiment, asingle linear line of sensor pixels is used to sense a one-dimensionaltrack of fingerprint features, and the signal sensed by the pixels isconverted from an analog signal to a digital signal, where the featuresare then represented as a string of digital values. For example, theridges of the fingerprint features may be represented as logical ones,and valleys represented as logical zeros.

When compared, the first string of digital values from one sample can becompared to the second string in a one to one relationship, and asimilarity score can be produced that measures the number of matchingvalues. If there is an immediate match, where both strings aresubstantially identical, then this would indicate that there was nomovement during the time between which the two samples were taken. Ifthere is not an immediate match, then this would indicate that there wassome movement, and additional comparisons may be needed to determine thedistance traveled. For each comparison, the strings of digital valuescan be shifted one or more pixels at a time. Once a good match is found,the distance traveled by the fingerprint is simply the number of pixelsshifted times the distance between the pixels, which may be measuredfrom the center point of one pixel to the center point of another pixelin the array of pixel sensors for example.

In one embodiment, a predetermined number of comparisons can be madealong with corresponding similarity scores. The process may then choosethe highest score to determine the most accurate comparison. The numberof pixels that were shifted to get the best comparison can then be usedto determine the distance traveled, since the size of and distancebetween the pixels can be predetermined, and the number of pixels canthus be used to measure the distance traveled by the fingerprint acrossthe motion sensor over the time period of the motion.

In another embodiment, the process could make comparisons and generatescores to measure against a predetermined threshold, rather than makinga predetermined number of comparisons. In this embodiment, thesimilarity score from each comparison can be measured after thecomparison is made. If the score is within the threshold, then it can beused to indicate the amount of shift from one sample to another. Thiscan then be used to determine the distance traveled by the fingerprintacross the linear motion sensor.

In one embodiment, generally, the invention provides a fingerprintmotion tracking system and method, where a single linear sensor array isconfigured to sense features of a fingerprint along an axis of fingermotion. The linear sensor array includes a plurality of substantiallycontiguous sensing elements or pixels configured to capture a segment ofimage data that represents a series of fingerprint features passing overa sensor surface. A buffer is configured to receive and store image datafrom the linear sensor array. And, a processing element is configured togenerate fingerprint motion data. The linear sensor array may beconfigured to repeatedly sense at least two substantially contiguoussegments of fingerprint data, and the processor can generate motion databased on at least two sensed contiguous segments of fingerprint data. Inoperation, the linear sensor array is configured to sense a first set offeatures of a fingerprint along an axis of finger motion and to generatea first set of image data captured by a plurality of substantiallycontiguous pixels of the sensor array. The linear sensor array is alsoconfigured to subsequently sense a second set of features of thefingerprint along an axis of finger motion and to generate a second setof image data captured by a plurality of substantially contiguous pixelsof the sensor array. The processing element can then compare first andsecond sets of image data to determine the distance traveled by thefingerprint over a time interval.

As used herein, linear sensor array is a generic term that relates to aportion of sensing elements, whether they are pixels in an opticalreader, a static or radio frequency reader that reads electric fieldintensity to capture a fingerprint image, piezoelectric components intouch-sensitive circuit fingerprint readers, or other elementsindicative of fingerprint readers, where the elements are used to sensea portion of the fingerprint, rather than the entire fingerprint. Suchsensor arrays may be configured in a number of ways within a matrix ofwell known sensor devices. For example, as previously discussed, severalmodern configurations are described and illustrated in pending U.S.Publication No. US 2006-0083411 A1 entitled: Fingerprint SensingAssemblies and Methods of Making; U.S. Publication US 2005-0244039 A1entitled: Methods and Apparatus for Acquiring a Swiped FingerprintImage; U.S. Publication US 2005-0244038 A1, entitled: FingerprintSensing Methods and Apparatus; U.S. Publication US 2003-0035570 A1entitled: Swiped aperture capacitive fingerprint sensing systems andmethods, and other applications that are all assigned to common assigneeValidity, Inc. Also, many other types of sensor matrices exist in theart directed to capturing fingerprint images. The invention is directedto a novel system, device and method that are not limited in applicationto any particular sensor matrix or array configuration. In fact, theinvention can be used in conjunction with or incorporated into suchconfigurations to improve performance, and further to reduce theprocessing resources required to capture and reconstruct images.

According to the invention, the linear sensor is substantiallycontiguous, which is to say that the sensor elements are in a relativeproximity to each other so that a first reading of a portion offingerprint features can be taken, followed by a second reading after ashort period of time from another position. The two samples can becompared to determine the relative distance traveled by the fingerprintsurface in relation to the sensor surface. The linear sensor isconfigured to merely take a relatively small sample of the fingerprintat one point in time, then another at a subsequent time. These twosamples are used to determine movement of the fingerprint. Two or moresamples maybe compared in order to compute direction and velocity of afingerprint surface relative to the linear sensing elements. Thesesamples may be linear, as described below and illustrated in thedrawings, so that a linear array of fingerprint features can be recordedand easily compared to provide a basis for motion, distance traveledover time. If more than one sensor is employed, it is possible todetermine direction of motion using vector addition with the differentlinear samples taken. Thus, some of the functions provided by theinvention are a result of taking a linear sample to give a basis forvector analysis. However, those skilled in the art will understand that,given the description below and the related drawings, other embodimentsare possible using other configurations of motion sensors, which wouldnot depart from the spirit and scope of the invention, which is definedby the appended claims and their equivalents, as well as any claims andamendments presented in the future and their equivalents.

One useful feature of the invention is that ambiguity in results issubstantially prevented. If properly configured, a system configuredaccording to the invention can consistently produce a result, where atleast two samples can be taken such that the features of one sampleoverlap with another sample. Then, comparisons can be made to determinethe amount of shift, indicating the amount of movement of thefingerprint across the linear sensor. In prior art systems and methods,it is often the case that no result occurs, and a singularity results.Thus, a user would need to rep eat sensing the fingerprint. In somesystems, substantial predictor algorithms have been created in anattempt to compensate or resolve the singularity when it occurs. Suchapplications are very large and demand a good deal of computation andprocessing resources, which would greatly bog down a portable device.According to the invention, sensing motion of a fingerprint issubstantially certain, where samples taken from the fingerprint surfaceare consistently reliable. This is particularly important in navigationapplications, where relative movement of the finger translates tomovement of an object such as a cursor on a graphical user interface(GUI), discussed further below.

In one embodiment, the linear sensor array may be used alone todetermine linear movement of a fingerprint. In another embodiment, thesingle sensor array may be used in conjunction with one or more otherlinear sensor arrays to determine movement in two dimensions. In eitherembodiment, the linear sensor arrays are utilized solely for determiningmotion. If the motion of the analyzed fingerprint occurs generally alonga predetermined axis of motion, the single linear sensor array can beutilized to sense the velocity of the fingerprint being analyzed. Tocapture and record the motion of a fingerprint that is not directedalong a predetermined axis of motion, two or more linear arrays (aplurality of arrays) can be used together to sense and record suchmotion, and a processor can determine the direction and speed of thefingerprint using vector arithmetic.

In yet another embodiment, one or more such linear arrays may be used inconjunction with a fingerprint sensor matrix to more accurately captureand reconstruct a fingerprint image. The sensor matrix can be configuredto sense and capture an image of a portion of a fingerprint beinganalyzed, and the one or more linear arrays can provide motioninformation for use in reconstructing a fingerprint image. A device soconfigured would be able to more accurately sense, capture, record andreconstruct a fingerprint image using less processing resources thanconventional devices and methods.

Alternatively, in yet another embodiment, one or more arrays can be usedto generate motion information for use in accurate navigationaloperations, such as for use in navigating a cursor on a graphical userinterface (GUI). Utilizing the improved processing functions of theinvention, an improved navigation device can be constructed that iscompatible with a portable device that has the power and processingrestrictions discussed above. Examples of such embodiments are describedand illustrated below.

A motion sensor configured according to the invention uses substantiallyless space and power compared to conventional configurations for motionsensing, navigation and fingerprint image reconstruction. Such aconfiguration can further provide aid to conventional fingerprintreconstructing processes by better sensing motion of a finger while itis being analyzed by a sensing device. This allows a fingerprint sensingdevice the ability to reconstruct a fingerprint analyzed by afingerprint sensor with reduced power. Utilizing the invention,conventional processes that need to match and construct fragmentedimages of a fingerprint, particularly devices that sense and process afingerprint in portions, can be optimized with information related tofingerprint motion that occurs while a fingerprint surface is beingread. Also, using this unique motion detection technology, optimalnavigation functions can be provided that demands significantly lesspower than conventional devices. Such navigation functions can enable alow power navigation device to be integrated in a portable devicesystem, such as a mouse pad used to move a cursor across a graphicaluser interface (GUI) on portable electronic devices including cellularphones, laptop computers, personal data assistants (PDAs), and otherdevices where low power navigation functions are desired. A novel systemand method are provided that uses minimal space and processing resourcesin providing accurate motion detection from which fingerprint sensors aswell as navigation systems can greatly benefit.

A device or system configured according to the invention can beimplemented as a stand alone navigation device, or a device to provideimage reconstruction information for use with a line imaging device thatmatches and assembles a fingerprint image. Such a line imaging devicemay be any imaging device configured to sense and capture portions of afingerprint, whether it captures individual perpendicular image lines ofa fingerprint, or multiple perpendicular lines. In operation, a motiondetection device can operate as a separate motion detection and/ordirection detection device. Alternatively, a motion detection device canbe used in conjunction with a line imaging device to more accurately andefficiently sense, capture, store and reconstruct a fingerprint image. Adevice configured according to the invention may include a single arrayof finger ridge sensing pixels or data sensor points centrally locatedalong the principal axis of motion to be detected, a sampling system toperiodically sample the finger contact across the array, and acomputational module or element that compares two sets of samplescollected at different times to determine the distance traveled whilebetween the two sample times. According to the invention, the motionsensor pixels do not necessarily need to have the same resolution as theline imager. The motion sensor pixels may in fact use a differentsensing technique than the imager.

Again, the invention provides separate operations for detecting motionand for sensing and capturing a fingerprint image. Thus, the techniquesused for the separate processes can be the same or may be differentdepending on the application. Those skilled in the art will understandthat different variations of the separate processes are possible usingknown techniques and techniques can be derived without any undueexperimentation. Such variations would not depart from the spirit andscope of the invention.

Devices and Applications for Improved Navigation and Control:

The same techniques used to derive finger location and speed to helpassist in assembling complete fingerprint images from partialfingerprint image scans can also be used for other purposes as well. Inanother embodiment of the present invention, these techniques can beused to create elegant “finger mouse” devices that allow the motion of auser's finger to control a computerized system in a manner similar tothat of a conventional computer “mouse”.

In this type of embodiment, the invention provides the capability ofdual axis finger motion sensing through additional finger motion sensingarrays. In this embodiment, there are two or more (a plurality of)sensor arrays for detecting motion, and each axis is independentlymeasured to determine the component of velocity in that axis. Thevelocity components from the individual axes are used to compute avector sum to determine the actual direction and velocity of motion ofthe finger with respect to the sensor surface. According to theinvention, it is not necessary to capture a full image of thefingerprint in order to determine the distance traveled and thevelocity. It is only necessary to capture either the finger location, orenough of a linear sample of fingerprint features along the line ofmotion of the fingerprint to allow motion to be computed.

In one embodiment, a plurality of samples, such as two or three samples,are captured by motion sensor pixels and are used to determine thedistance traveled across the axis of motion of the fingerprint relativeto the sensor surface and the velocity at which the motion occurs. Thisinformation can also be used in “mouse like” computerized devicenavigational operations. If desired, the information can also of course,further be used in combination with a fingerprint imager to aid inreconstructing a fingerprint image.

In order to provide a navigation device, as well as to detect andcorrect for finger motion that is not completely aligned with thedesired axis, either of the embodiments may be combined in ensemblessuch that one sensor is aligned on the axis of motion, and additionalsensors aligned at an angle (such as 22.5 or 30 degrees) to theprincipal axis of finger motion. Examples of different embodiments arediscussed below.

Referring to FIG. 2A, a diagrammatic view of motion detection andtracking system configured according to the invention is illustrated. Anintegrated circuit package (100) or other unitized electrical componentis illustrated having circuits and possibly software embedded (notshown) and electrical connections (101) for integration in andconnection with a system that utilizes the circuit package. FIG. 2Aillustrates an embodiment of the invention where a finger (104) can moveits fingerprint surface (106) against sensor surface (108) to be read bythe sensors (110), (112). These sensors can pick up movement informationof a fingerprint for use in navigational applications, or can be used inconjunction with an integrated fingerprint sensor surface (108) tosimultaneously capture and record portions of a fingerprint. Such asystem configured according to the invention may be a stand alonecomponent as shown, or can be integrated with other circuits for morespace and power savings as well as efficiency. In a preferredembodiment, the sensor used to detect finger motion and the sensor usedto image at least part of the fingerprint are present on the sameunitized electrical component so that the two types of sensors aremounted or dismounted from an electrical appliance or device as a singleunit. Those skilled in the art will understand that many variations ofthe configuration are possible, and that the invention is not limited toany particular configuration, but is defined by the claims and allequivalents.

The system further includes a sensor module (102) that is used to sensea user's finger (104) and fingerprint surface (106) when it is movedacross fingerprint sensing surface (108). As can be seen, thefingerprint sensing surface (108) is illustrated as a narrow surfacethat is designed to sense and capture portions of a fingerprint as it ismoves across the sensor. These portions can be subsequentlyreconstructed according to the invention using motion information fromthe motion sensors (110), (112). Thus, the sensor components illustratedin FIG. 2A have multiple utilities, and can be configured in devicesthat utilize part or all of such utilities, whether it is a stand alonemotion sensor configured to sense movement and velocity in onedirection, a multidirectional motion sensor configured to sense movementand velocity in several directions, or a combination device configuredto sense motion either in one or more (one or more meaning a pluralityof directions) directions and used in combination with a fingerprintsensor surface that reads portions of fingerprints and reassembles thefingerprints using the motion information from motion sensors.

Referring to FIG. 2B, a side view of the unitized sensor system of FIG.2A is illustrated. In operation, the finger (104) is placed by a useronto the sensor surface (107), which includes fingerprint sensingsurface (108), so that the fingerprint sensing surface (108) and thesensor surface (106) are juxtaposed relative to each other. The finger(104) may move in direction “A” and the sensor (100) may remainstationary. Alternatively the finger (104) and the sensor (100) may bemoved in opposite directions A, B. Or the finger may be moved from sideto side, or some combination of finger movements may be performed. Ineither case, device (100) will pick up the finger motions and translatethese motions into electrical guiding impulses which may be used tocontrol an electronic device. Thus device (100) can be considered to bea “finger mouse chip”. This “finger mouse chip” may be placed on a widevariety of different electronic devices, and used to control a widevariety of functions, in the same way that a conventional mouse orjoystick or mouse pad controls functions.

As an example, in FIG. 2C, chip (100) is embedded into a portablemicroprocessor controlled device (202). Device (202) could be a portablemusic player, a cellular phone, PDA or other device. In this example,device (202) has a graphical user interface (GUI) or screen (204), and acursor (206) that may app ear on the screen. This cursor is capable ofbeing moved across the screen under control of a user navigating atouch-sensitive cursor control (208), which in some embodiments issimply the “finger mouse chip” (100). The touch sensitive cursor hasnavigational indicia (210), which may be merely directional indicatorslocated about sensor (102) that is located within or about thattouch-sensitive cursor that acts as a navigational pad, similar to thatof a mouse pad commonly used on laptop computers.

According to the invention, such a navigational pad can be greatlyenhanced using sensor technology according to the invention, wheredirectional movement sensors (110), (112) are used to guide the cursor(206) for searching for and selecting indicia such as toolbar items oricons for opening files, photos and other items when selected. In someapplications, a multi-step sensor can read the fingerprint structuresfor guidance at one level, and may select indicia by pressing harder onthe sensor for another level of sensing, Thus, a user can move thecursor around by lightly pressing on and moving a finger along thesurface, then pressing harder when selecting an icon, toolbar or otherindicia. Utilizing the invention, a more efficient navigation tool canbe adapted to perform all of these tasks at low power and high accuracy,a very adaptable feature for portable devices.

Referring to FIG. 2D, another embodiment of the invention is illustratedwhere the integrated circuit (IC) chip 116 is separate from the sensorsurface 108(b). This configuration is particularly feasible for thefinger position sensors of the present invention which are based on deepfinger penetrating radio frequency (RF) technology. (See U.S. Pat. Nos.7,099,496; 7,146,024; and U.S. patent application Ser. Nos. 11/107,682;11/112,338; 11,243,100; and 11/184,464, the contents of which areincorporated herein by reference). In particular, see the flexiblecircuits of application Ser. No. 11/243,100, (for example FIG. 3), thecontents of which are incorporated herein by reference. See also presentFIG. 1I.

In this case, as previously discussed for FIG. 1I, the basic chip sensorcircuit (see FIG. 1A (116)) may be mounted on a flexible thin support orfilm, and electrical lines may then extend out form the chip to form thevarious electrical sensing arrays. Usually electrical traces (oralternatively fiber optics) will at least partially connect the actualsensing arrays (110), (112) with the IC chip (116). However if deepfinger penetrating radio frequency (RF) technology or optical sensingtechniques are used, these traces need not be completely continuous. Forsimplicity, these traces are not shown.

Referring again to FIG. 2A, the surface (108) has embedded finger motionsensors (112) that, according to the invention, operate to detect thepresence and motion of a fingerprint surface (106) about the sensorsurface (108). These sensors can include a single motion sensor (110),aligned with a general finger motion direction for detecting distancetraveled by the finger across the sensor over a period of time. Thisallows a processor to compute the velocity of the finger over the sensorsurface. In another embodiment, there may be a single motion sensor(110) on the surface (108), or there may be a plurality, two or moremotion sensors (110), (112), on the surface (108), depending on theapplication. The additional sensors (112) may be used to detectdirection of a finger's motion across the sensor surface. In practicalapplications, a user may not move the finger exactly parallel with thesensor (110). A user may rub the finger surface (106) at an angle withrespect to the axis of the sensor (110). A processor analyzing thevelocity of the finger motion may then end up with an inaccuratevelocity reading. This may be important when the data generated by thesensor is used for reconstructing a fingerprint, or when the sensor datais used for navigational purposes. According to this additionalembodiment of the invention, the additional sensors (112) can be used todetermine the direction of the finger surface when it is being analyzed.Using the data captured by the sensors, a processor can apply vectoranalysis to generate motion information. This motion information can beused in processes for reconstructing the fingerprint images, oralternatively for “mouse like” navigation processes.

One advantage of having motion sensors and possibly fingerprint imagersarranged to accept fingers moving with two dimensions of freedom (thatis, combinations of up and down and right and left), is thatfingerprints can now be created from very different types of fingerswipes. As shown in FIG. 2E, a fingerprint image may be created from aseries of partial fingerprint images created with a finger swipe goingin the vertical direction. By contrast, FIG. 2F shows that a fingerprintimage may be created from a series of partial fingerprint images with afinger swipe going in the horizontal direction. By combining fingermotion sensors and partial fingerprint scanners aligned in more than onedirection on the same device, users may use the device accurately undera much broader range of operating conditions, and thus may view suchdevices has having greatly superior performance and usability.

FIGS. 3-7 discussed below have a similar numbering pattern where thesensor surface (107) includes the two other sensing surfaces: anoptional fingerprint sensing surface (108) and finger motion sensors(110) and (112). The different finger motion sensing devices, whetherincluded with an image sensor for sensing a partial fingerprint imagefor future full fingerprint reconstruction, can be utilized fornavigational operations, either with the partial fingerprint imager, oron a stand alone basis where the partial fingerprint imager is eitherturned off or is absent. In a preferred embodiment, the various fingermotion sensors and the various partial fingerprint image sensors arepresent on the same unitized device, where by the two types of sensorscan be simultaneously used as a single component of a larger electricalappliance. These different embodiments are described below in relationto sensing, capturing and reconstructing fingerprint images, but arealso applicable in providing motion and direction information for use asnavigational information, such as for use in navigating a cursorrelative to the motion of a fingerprint over motion sensors.

According to another embodiment 102(a) of the invention illustrated inFIG. 3, the sensor surface 108(a) may include image sensing elementsused for broadly sensing and recording the fingerprint features. Inaddition, a motion sensor 110(a) is included for sensing and recordingthe motion of the fingerprint. Such a device may be a single sensorembedded within the two dimensions of the sensor surface 107(a), withthe fingerprint sensing surface 108(a) included for sensing andrecording the full fingerprint. The motion sensors are configured toseparately sense and recording motion information. Here, the sensorsurface 107(a) includes a motion sensor 10(a) configured separately fromfingerprint sensing surface 108(a). According to this embodiment, themotion sensor is separate from the fingerprint sensing surface, thoughlocated on the same sensor surface. In operation, a fingerprint surface106 can be moved simultaneously along motion sensor 10(a) andfingerprint sensing surface 108(a). The motion information from themotion sensor, such as distance and time traveled over that distance,can be utilized together with the fingerprint sensing surface as an aidin reconstructing the separate portions of the fingerprint. As describedfurther below, such a single motion sensor can also be used fornavigation functions as well.

Referring to FIG. 4, another embodiment 102(b) of the invention isillustrated where motion sensors 10(b), 112(b) are located aboutfingerprint sensor surface 108(b) within sensor surface 107(b). Themotion sensor 110(b) is located along an anticipated axis of motion offinger 106 with respect to device 100 in directions A, B. Motion sensor110(b) can sense the distance and time expended over that distance todetermine velocity, which can be used in reconstructing the fingerprintportions simultaneously captured by fingerprint sensor surface 108(b).Using the additional motion sensors 112(b), a fingerprint surface 106can be sensed and captured even if a user slides the finger at an angleto the axis of the motion sensor 110(b). In fact, given the angles ofthe additional sensors 112(b) with respect to the central axis of thedevice, the direction of motion can be computed by a processor usingvector addition. Thus, the direction, distance and time expended duringfingerprint surface travel across the sensors can be used along with thefingerprint portions captured by the fingerprint sensor to accuratelyreconstruct the fingerprint image.

Referring to FIG. 5A, yet another embodiment 102(C) of the invention isillustrated, where the finger motion sensors 110(C), 112(C) areinterleaved with optional fingerprint sensor surface 108(C) in acombined component within sensor surface 107(C). Such a configurationcan be created in a sensor surface, where the pixels or data contactpoints that sense the fingerprint features are separately read from thesensors by a processor. For example, in a matrix of sensor pixels ordata contact points, individual points can be singled out in one or morearrays to operate as motion sensing arrays. In the same matrix, theremaining pixels or data contact points can form a fingerprint sensorsurface for sensing and capturing the fingerprint image. In operation, afingerprint can be juxtaposed and moved along the sensor surface 107(C)along the anticipated axis of motion or at another angle, and anaccurate sense and capture of a fingerprint can be achieved withoutundue computation and power load. While the fingerprint sensor surface108(C) senses and captures the portions of images of the fingerprintfeatures upon contact with the fingerprint surface 106, the motionsensors can simultaneously capture finger motion information as thefeatures move past the motion sensors. The motion information can beused in combination with the portions of fingerprint images toreconstruct the fingerprint image. Alternatively, the fingerprint sensor108(C can be absent or turned off, and the device used for “fingermouse” navigational purposed only.

Referring to FIG. 5B, the same configuration of FIG. 5A is illustrated,with a view of the motion sensors shown much smaller in comparison tothe overall sensor surface. In a sensor surface that is denselypopulated with pixels or data contact points, the relative size of theportion of the sensor surface that is covered with the motion sensingarrays are very small compared to the pixels and data points that makeup the fingerprint sensing surface 108(C), both located within sensorsurface 107(C). Thus, the fingerprint can be sensed and captured withoutany interference by the interleaved motion sensing arrays and accurateportions of a fingerprint image can be captured and accuratelyreconstructed using the combined information from the fingerprintsensors and the motion sensors. Utilizing this embodiment, a universalcomponent can be constructed and utilized for both motion detection andfingerprint capture. These motion sensors, which can sense both motionand direction, can also be used for navigation operations.

Referring to FIG. 6, another embodiment 102(d) of the invention isillustrated, where a single motion sensor array 110(d) is interleavedwithin the fingerprint sensor surface 108(d) of sensor surface 107(d).Unlike the embodiment illustrated in FIGS. 5 a, 5 b, this embodiment islimited to one motion sensor array located along the anticipated axis ofmotion of the finger, which is anticipated to move in directions A, Bwith respect to the device 100. In operation, the interleaved sensorarray 110(d) can sense and capture motion information regarding themotion of the finger across the sensor surface 107(d), whilesimultaneously fingerprint sensor surface 108(d) can sense and capturethe fingerprint images for subsequent reconstruction. The informationfrom both sensors can be used to more accurately reconstruct thefingerprint image. The information of both motion and direction can alsobe used for navigation operations.

Referring to FIG. 7, yet another embodiment 102(e) of the invention isillustrated, where multiple motion sensors 112(e) are interleaved withinfingerprint sensor surface 108(e). This embodiment is similar to thatillustrated in FIGS. 5 a, 5 b, but with more motion sensors at variousangles. In operation, a fingerprint can be juxtaposed and moved alongthe sensor surface 107(e) along the anticipated axis of motion or atanother angle, and an accurate sense and capture of a fingerprint can beachieved without undue computation and power load. While the fingerprintsensor surface 108(e) senses and captures the portions of images of thefingerprint features upon contact with the fingerprint surface 106, themotion sensors can simultaneously capture motion information as thefeatures move past the motion sensors. The motion information can beused in combination with the portions of fingerprint images toreconstruct the fingerprint image.

If used for navigation purposes, of the motion sensor configurationsabove can be utilized for different navigation operations. For example,referring again to FIG. 3, the motion sensor 110(a) can be utilized onits own to sense motion in one axis of motion, for example in onedirection. One application may be a sensor used for a power, volume orother audio control, where an up or down motion can be used to adjustthe power, volume or other audio value. Here the motion sensor 110(a)can be used on a stand alone basis, and the fingerprint scanner 108(a)can be either turned off or absent.

Another application for the invention is the implementation of a scrollfunction for lists of data or text in a GUI. Precise power control overa range may be useful in manufacturing environments, where small changesin power can greatly affect a process. Another application may be tooperate a medical instrument where accuracy is useful to the device'soperation.

Computer-mouse-like navigation requires ability to sense motion in twodimensional space, where motion and direction information are required.Referring again to FIG. 4, a separate motion sensor 110(b) isillustrated for individual sensing of motion and direction, wheredistance, time expended over the distance (allowing for calculation ofvelocity), and direction can be calculated. Though this motioninformation can be used to enable better processing and reconstructionof fingerprint images as discussed above, as previously discussed, itcan be used separately for navigation, making it a navigation sensor. Inoperation, the separate motion sensor can detect motion and direction,giving information required for navigation functions. In operation, anavigation sensor can consistently computing the matches for the variousaxes, generating motion and direction information as a fingerprint movesabout a sensor.

Thus, if a user would stroke a fingerprint surface against a motionsensor surface, the arrays could pick up the motion and directioninformation, and a processor could process the information to generaterelative motion and direction information for use in navigation, such asfor a computer mouse. In this example, a user can move a finger relativeto a cursor on a graphical user interface (GUI), such as a computerscreen, a cellular phone, a personal data assistant (PDA) or otherpersonal device. The navigation sensor could then cause the cursor tomove relative to the fingerprint motion, and a user can navigate acrossthe GUI to operate functions on a computer or other device. Since themotion of the cursor is relative to the movement of the fingerprintsurface against the navigation sensor, relatively small movements cantranslate to equal, lesser or even greater distance movement of thecursor.

Referring to FIG. 8A, another embodiment of the invention isillustrated, where multiple arrays are located on the sensor surface toallow for sensing and capturing motion and direction information indifferent directions of fingerprint travel for use in navigationapplications and other applications. The base film or circuit (120),which may be a flexible electronic circuit other material, includes asensor surface (121) having several motion sensor arrays. Suitablematerials for this base film or circuit were described in applicationSer. No. 11/243,100, the contents of which are incorporated herein byreference. The overall concept here is similar to the three sensor arraypreviously illustrated in FIG. 5A.

In the present example, there are three sensors that fan upward fordetecting motion and direction. In operation, a user typically willstroke over the sensor in a downward direction, and the three sensorscan determine the direction and speed using vector analysis. However, itmay be desired to account for motion in either an upward or downwarddirection, and multiple sensors in either direction would be useful tobetter capture the information. From an orientation of a user facing thesensor illustrated in FIG. 8( a), the right sensors (122), (124) facethe right, and are configured to capture movement toward the right,where either sensor could capture movement motion from the upper rightto the lower left, and from the upper left to the lower right. Sensors(126), (128) could capture up or down movement, and sensors (130), (132)face the left, and are configured to capture movement toward the right,where either sensor could capture movement motion from the upper rightto the lower left. Utilizing the multiple sensors, a sensor would bemore robust, capable of sensing more fingerprint features, and also ableto process more movement and directional information for use incapturing and reconstructing fingerprint images or for otherapplications such as navigation. The angle occurring between sensor(121) and center horizontal line (134) can be any angle, such as 30, 45or 22.5 degrees in order to most effectively capture movement that isnot aligned with center sensors (126), (128). All off-axis sensors(124), (128), (130), (132) can be set at various angles, which candepend on a particular application.

Referring to FIG. 8B, an even more robust example of a sensor set onflexible circuit (136) having a surface (137) located on the circuit.The sensor (138) is located on the circuit surface (137), and includesmultiple array sensors (140) that are set at various angles. In thisembodiment, each array may be set at 22.5 degrees from adjacent angles,providing a wide variety of angles at which to sense and capture motioninformation. The sensor, similar to that of FIGS. 8( a) and 2B, has anIC chip (139) that is separate from the sensor surface (138), similar tothe configurations previously discussed in application Ser. No.11/243,100, the contents of which are incorporated herein by reference.

Referring to FIG. 8C, a diagrammatic view of multiple array sensorslocated on a sensor (142) is illustrated. Sensors (144), (144′) arevertical arrays that are set to capture one axis of motion. Sensors(146), (146′) and (150), (150′) are located off axis at an angle tosensors (144), (144′). Sensors (148), (148′) are optional and may beused in conjunction with the other sensors to gather motion informationin a horizontal direction with respect to the vertical sensors. Inpractice, either or all of these sensors can be utilized by a system toaccurately sense and capture motion and direction information inmultiple directions. Again, which sensors to use may depend on aparticular application and configuration.

In one embodiment, in order to support motion at any arbitrary angle,sensor arrays may be oriented at approximately 0, 30, 60, 90, 120, and150 degrees. Another more robust system might space them at 22.5 degreeincrements, rather than 30. Once motion reaches 180 degrees, the processcan use reverse motion on the zero degree sensor array, and so on. Aspreviously discussed, a device configured in this way would have some ofthe properties of a navigation touchpad such as those used in laptopcomputers, with the relative motion sensing capability of a computermouse.

Such finger motion sensors will often be deep finger penetrating radiofrequency (RF) based sensors, as previously discussed in U.S. Pat. Nos.7,099,496; 7,146,024; and U.S. patent application Ser. Nos. 11/107,682;11/112,338; 11,243,100; 11/184,464; however alternative sensingtechniques (optical sensors, etc.) may also be used. The circuitry usedto drive such sensors was previously shown in partial detail in FIG. 1A,and is now shown with additional detail in FIG. 9.

As before, the overall device (100) is one or more sensor elementscomposed of linear arrays of finger position sensing plates, and otheroptional sensors such as fingerprint imaging sensors. The fingerposition sensing plates will usually be activated or scanned by sensorcontrol logic (252), which will send electrical signals, to the sensingplates in some sort of rapid scanning order. Control logic (252) mayalso control power, reset control of the sensor pixels or data contactpoints, control the output signal, control light sources or cooling (ifan optical sensor is used), or perform other standard control functions.The output from these plates will in turn be detected by a readoutcircuit (254). This readout circuit is usually controlled by anamplifier (256) to detect and amplify the electrical signal from aparticular plate. This signal is normally affected by the presence orabsence of a finger. The output from amplifier (258) will often then befiltered with a low pass filter (258) to reduce ambient electricalnoise, and will normally then be digitized by an analog to digitalconverter (260). This data will then normally be transmitted by acommunications link, such as a system bus (280), serial link, parallellink, or some other data transmission means to other processing devicesfor further analysis.

The readout circuit (254) may store the output signal (data) in storage(262). If fingerprint images are obtained, the fingerprint data (264) isstored and preserved, either temporarily until the processor (266) canprocess the data, or for later use by the processor as needed. Theprocessor (266) includes arithmetic unit (268) configured to processalgorithms used for navigation of a cursor, such as those described inconnection with navigation features of FIG. 2B, and for optionalreconstruction of fingerprints. Processing logic 270 is configured toprocess information and includes other logic utilized by a processor.Persistent memory (274) is used to store algorithms (276) and softwareapplications (278) that are used by the processor for the variousfunctions described above, and in more detail below. Softwareapplications (278) can include applications that use the finger movementinformation provided by the finger position sensors to control variouscomputerized functions, such as moving a cursor, activating menus,controlling system parameter settings, and the like.

Referring to FIG. 10, a flow chart (1000) of one embodiment of anavigation sensor operation algorithm is illustrated. The process beginsat step (1002), and, in step (1004), motion information is received,such as distance, time and velocity information. In step (1006),direction information is received from the sensors. In step (1008),relative motion for navigation is calculated by a processor. In step(1010), direction information for navigation is calculated. And, in step(1012), navigation operations are performed. The process ends at step(1014). This algorithm will often be stored in persistent memory FIG. 9,(276).

More specifically, the navigation sensor operation algorithm can be usedas is a finger mouse cursor control algorithm and this more specificcase is shown in FIG. 11 (1100). As before, the process begins in step(1102), and in step (1104) finger motion information is received, suchas finger distance, time and velocity. In step (1106), finger directioninformation is received. In step (1108), relative motion and directionfactors are calculated for use in operating the cursor. In step (1110),the cursor is moved according to the relative motion and directionfactors calculated in step (1110). The process ends in step (1112).

The invention may also involve a number of functions to be performed bya computer processor, such as a microprocessor. The microprocessor maybe a specialized or dedicated microprocessor that is configured toperform particular tasks by executing machine-readable software codethat defines the particular tasks. The microprocessor may also beconfigured to operate and communicate with other devices such as directmemory access modules, memory storage devices, Internet relatedhardware, and other devices that relate to the transmission of data inaccordance with the invention. The software code may be configured usingsoftware formats such as Java, C++, XML (Extensible Mark-up Language)and other languages that may be used to define functions that relate tooperations of devices required to carry out the functional operationsrelated to the invention. The code may be written in different forms andstyles, many of which are known to those skilled in the art. Differentcode formats, code configurations, styles and forms of software programsand other means of configuring code to define the operations of amicroprocessor in accordance with the invention will not depart from thespirit and scope of the invention.

Within the different types of computers, such as computer servers, thatutilize the invention, there exist different types of memory devices forstoring and retrieving information while performing functions accordingto the invention. Cache memory devices are often included in suchcomputers for use by the central processing unit as a convenient storagelocation for information that is frequently stored and retrieved.Similarly, a persistent memory is also frequently used with suchcomputers for maintaining information that is frequently retrieved by acentral processing unit, but that is not often altered within thepersistent memory, unlike the cache memory. Main memory is also usuallyincluded for storing and retrieving larger amounts of information suchas data and software applications configured to perform functionsaccording to the invention when executed by the central processing unit.These memory devices may be configured as random access memory (RAM),static random access memory (SRAM), dynamic random access memory (DRAM),flash memory, and other memory storage devices that may be accessed by acentral processing unit to store and retrieve information. The inventionis not limited to any particular type of memory device, or any commonlyused protocol for storing and retrieving information to and from thesememory devices respectively.

The apparatus and method include a method and apparatus for enabling andcontrolling fingerprint sensors and fingerprint image data and motiondata in conjunction with the operation of an electronic device wherenavigation and fingerprint verification processes are utilized. Althoughthis embodiment is described and illustrated in the context of devices,systems and related methods of imaging fingerprints and navigationfeatures for a portable device, the scope of the invention extends toother applications where such functions are useful. Furthermore, whilethe foregoing description has been with reference to particularembodiments of the invention, it will be appreciated that these are onlyillustrative of the invention and that changes may be made to thoseembodiments without departing from the principles of the invention.

1. A finger motion tracking apparatus used to provide user input to acomputing_device, comprising: at least two linear sensor arrays eachconfigured to sense overlapping line segments of features of afingerprint of a finger moving along a one of at least two axes ofmotion of the finger, each linear sensor array including a plurality oflinear sensor array sensing elements configured to capture segments offingerprint image data, at least one of the at least two axes of motionbeing in a direction other than a direction of a swipe of the fingersurface over the sensor surface, the overlapping line segments of thefeatures of the fingerprint image each being sensed at different times;a memory configured to receive and store image data from each linearsensor array; and a computing device configured to generate fingermotion data by comparing overlapping line segments of the fingerprintimage that match each other, sensed by one of the at least two linearsensor arrays at different times, thereby determining a velocity offinger motion along each of the at least two axes of motion of thefinger; wherein the at least two linear sensor arrays are each presenton the same unitized electrical component.
 2. The apparatus of claim 1,wherein the apparatus is configured to use data collected from at leastone of the at least two finger position sensor arrays to control thecreation of a complete fingerprint image from partial fingerprint imagescollected using at least one partial fingerprint sensor.
 3. Theapparatus of claim 2, wherein the at least one finger position sensorarray and the at least one partial fingerprint imager are driven by thesame integrated circuit chip.
 4. The apparatus of claim 2, wherein atleast one of the at least one finger position sensor array or the atleast one of the partial fingerprint imager are deep finger penetratingradio frequency (RF) based sensors.
 5. The apparatus of claim 2, whereinthere are a plurality of the at least one partial fingerprint imagers,and in which the orientation of at least some of the partial fingerprintimagers are different from the others, whereby when a finger is swipedat an arbitrary two dimensional angle an adequate fingerprint image isobtained.
 6. The apparatus of claim 2, wherein both the at least onefinger position sensor array and the at least one partial fingerprintimager are located on the same flexible support, and an integratedcircuit that drives at least some of the finger position sensor arraysand at least some of the partial fingerprint imagers is located on thesame flexible support.
 7. The apparatus of claim 2, in which outputsfrom the apparatus include at least two spatial dimensions of fingerposition or finger motion data, and the partial fingerprint images fromat least one partial fingerprint imager.
 8. The apparatus of claim 7,wherein at least one of the finger position sensor arrays is a lineararray of electrically conducting plates, and in which at least one ofthe partial finger print imagers produces a one dimensional partialimage of a fingerprint.
 9. The apparatus of claim 1, wherein there are aplurality of the at least one finger position sensor arrays, and inwhich the orientation of at least some of the finger position sensorarrays are different from the others, whereby two dimensional fingermotion or location information is obtained.
 10. The apparatus of claim1, in which outputs from the apparatus include at least two spatialdimensions of finger position or finger motion data.
 11. A finger motiontracking apparatus used to provide a user input to_a computing devicecomprising: at least two linear sensor arrays configured to senseoverlapping line segments of features of a fingerprint of a fingermoving along one of at least two axes of motion of the finger, eachlinear sensor array including a plurality of linear sensor array sensingelements configured to capture segments of fingerprint image data, atleast one of the at least two axes of motion being in a direction otherthan a direction of a swipe of the finger surface over the sensorsurface, the overlapping line segments of the features of thefingerprint image each being sensed at different times; a memoryconfigured to receive and store image data from each linear sensorarray; and a computing device configured to generate finger motion databy comparing overlapping line segments of the fingerprint image thatmatch each other, sensed by one of the at least two linear sensor arraysat different times, thereby determining a velocity of finger motionalong each of the at least two axes of motion of the finger.
 12. Anapparatus according to claim 11, wherein the linear sensor array isconfigured to sense at least two overlapping line segments offingerprint data, including a first overlapping line segment and asubsequently sensed second overlapping line segment; and wherein thecomputing device is configured to generate motion data based on anamount of shift between the first overlapping line segment and thesubsequently sensed second overlapping line segment and navigation databased on the amount of shift in each of the at least two axes.
 13. Anapparatus according to claim 12, wherein the user input moves a cursoron a graphical user interface.
 14. An apparatus according to claim 11,wherein the motion data includes velocity data and directional data. 15.The apparatus according to claim 11, wherein the linear sensor array isconfigured to sense a first set of features of a fingerprint along afirst axis of finger motion and to generate a first set of image datacaptured by a plurality of sensors in the linear sensor array and tosubsequently sense a second set of features of the fingerprint along thefirst axis of finger motion and to generate a second set of image datacaptured by the plurality of sensors in the linear sensor array, andwherein the computing device is configured to compare the first set andthe second set of image data to determine the distance traveled by thefinger over a time interval, and wherein the distance and time intervalare used to calculate navigational data for navigating the cursor on thegraphical user interface.
 16. The apparatus according to claim 11,further comprising the second of the at least two linear sensor arraysconfigured to sense features of the fingerprint along the second of theat least two axes and to generate directional image data captured by aplurality of sensors, wherein the computing device is configured toproduce navigation data using the motion data along the second of the atleast two axes of motion.
 17. The apparatus according to claim 11,further comprising at least a second linear sensor array aligned along asecond axis of the at least two axes of motion and configured to sensefeatures of a fingerprint and to generate directional image datacaptured by a plurality of sensors for use in providing user input forthe navigation of an object responsive to the motion of a finger acrossthe sensor surface.
 18. The apparatus according to claim 11, wherein thecomputing device is configured to use at least one of the motion dataand the navigation data to assemble fingerprint image portions.
 19. Amethod of tracking motion of a fingerprint with respect to a sensorsurface for use in a user input to computing device comprising: sensingat least two overlapping line segments of a fingerprint image, eachlocated along one of at least two axes of motion of the finger surfacewith respect to the sensor surface, at least one of the at least twoaxes of motion being in a direction other than a direction of a swipe ofthe finger surface over the sensor surface, the overlapping linesegments of the fingerprint image each being sensed at different times;storing digital data corresponding to the at least two overlapping linesegments sensed by the sensing elements; and processing the digital datato generate fingerprint motion data by comparing overlapping linesegments of the fingerprint image that match each other, sensed by oneof the at least two linear sensor array at different times, therebydetermining a velocity of finger motion along each of the at least twoaxes of motion of the finger, and navigation data based on the amount ofshift in each of the at least two axes.
 20. A method according to claim19, further comprising: comparing digital data of a first overlappingline segment with subsequently sensed digital data of a secondoverlapping line segment; determining the amount of shift between thefirst overlapping line segment and the subsequently sensed secondoverlapping line segment to determine the distance traveled by thefinger across the sensor surface in the direction of the one of the atleast two axes; and generating navigation data from the amount of shift.21. A method according to claim 20, further comprising: estimating anestimated distance traveled by the finger surface with respect to thesensor surface in the direction of the one of the at least two axes bymultiplying the amount of pixel shift detected times the pitch of thesensor pixels in the line segment of the sensor; and computing thevelocity of the finger surface with respect to the sensor surface bydividing the estimated distance by a time between the sensing of thefirst overlapping line segment and the second overlapping line segmentin the direction of the one of the at least two axes.
 22. A methodaccording to claim 20, further comprising: estimating a distancetraveled by the finger surface with respect to the sensor surface bymultiplying the pixel shift detected between the first overlapping linesegment and the second overlapping line segment in the direction of theone of the at least two axes times the pitch of the pixels in thedirection of the one of the at least two axes; and computing thevelocity of the finger surface with respect to the sensor surface bydividing the estimated distance by the time between the sensing of thefirst overlapping line segment and the sensing of the second overlappingline segment.
 23. A tangible machine readable medium storinginstructions that, when executed by a computing device, cause thecomputing device to perform a method, the method comprising: trackingmotion of a fingerprint with respect to a sensor for use in a user inputto a computing device comprising: sensing at least two overlapping linesegments of a fingerprint image, each located along one of at least twoaxes of motion of the finger surface with respect to a sensor surface,at least one of the at least two axes of motion being in a directionother than a direction of a swipe of the finger surface over the sensorsurface, the overlapping line segments of the fingerprint image beingsensed at different times; storing digital data corresponding to the atleast two overlapping line segments sensed by the sensing elements; andprocessing the digital data to generate fingerprint motion data bycomparing overlapping line segments of the fingerprint image, that matcheach other, taken by one of the linear sensor arrays at different times,thereby determining a velocity of finger motion along each of the atleast two axes of motion of the finger, and navigation data based on theamount of shift in each of the at least two axes.
 24. The machinereadable medium of claim 23, the method further comprising: comparingdigital data of a first overlapping line segment with subsequentlysensed digital data of a second overlapping line segment; determiningthe amount of shift between the first overlapping line segment and thesubsequently sensed second overlapping line segment to determine thedistance traveled by the finger across the sensor surface in thedirection of one of the at least two axes; and generating navigationdata from the amount of shift.
 25. A machine readable medium of claim24, the method further comprising: estimating an estimated distancetraveled by the finger surface with respect to the sensor surface in thedirection of one of the at least two axes by multiplying the amount ofpixel shift detected times the pitch of the sensor pixels in the linesegment of the sensor; and computing the velocity of the finger surfacewith respect to the sensor surface by dividing the estimated distance bya time between the sensing of the first overlapping line segment and thesensing of the second overlapping line segment in the direction of oneof the at least two axes.
 26. A machine readable medium of claim 24, themethod further comprising: estimating a distance traveled by the fingersurface with respect to the sensor surface by multiplying the pixelshift detected between the first overlapping line segment and the secondoverlapping line segment in the direction of the one of the at least twoaxes times the pitch of the pixels in the direction of the one of the atleast two axes; and computing the velocity of the finger surface withrespect to the sensor surface by dividing the estimated distance by thetime between the sensing of the first overlapping line segment and thesensing of the second overlapping line segment.